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Abstract 

This  paper  presents  the  design  of  a  vision-based  con¬ 
troller  for  an  underactuated,  unmanned  aerial  vehicle 
(UAV)  equipped  with  a  pan-tilt  camera  unit  (PTCU)  to 
achieve  the  objective  of  following  a  leader  vehicle  au¬ 
tonomously.  The  relative  position  and  orientation  in¬ 
formation  is  obtained  from  the  monocular  camera  uti¬ 
lizing  homography-based  techniques.  The  proposed  con¬ 
troller,  built  upon  Lyapunov  design  methods,  achieves 
uniform  ultimate  bounded  (UUB)  tracking.  As  an  ex¬ 
tension,  it  is  also  demonstrated  that  the  approach  used 
in  the  development  of  the  control  strategy  for  the  leader- 
follower  problem  can  be  applied,  with  a  few  modifica¬ 
tions,  to  the  problem  of  trajectory  tracking,  where  the 
desired  trajectory  is  described  as  a  sequence  of  images 
taken,  for  example,  by  the  on-board  camera  during  a 
previous  flight. 

1  Introduction 

The  advantages  of  employing  UAVs  in  dangerous  and  un¬ 
structured  environments  have  already  made  them  an  es¬ 
sential  component  in  various  civilian  tasks  such  as  disas¬ 
ter  related  search  and  rescue,  terrain  mapping  and  aerial 
surveillance,  as  well  as  in  modern  warfare.  Fueled  by  the 
recent  availability  of  low-cost  hardware  such  as  remotely 
radio-controlled  electric  and  gas  powered  helicopters  and 
fixed-wing  aircrafts,  numerous  research  groups  are  currently 
working  towards  developing  novel  applications  and  control 
strategies  for  UAVs.  Among  the  most  popular  is  the  quad- 
rotor  helicopter  design,  chiefly  due  to  its  design  simplicity 
and  high  maneuverability.  New  studies  aimed  at  develop¬ 
ing  high-performance  quad-rotor  helicopters  have  appeared 
in  recent  publications  [22],  and  provides  insights  to  vari¬ 
ous  stability  problems  and  control  challenges  involved  [18]. 
These  vehicles  are  underactuated,  i.e.,  they  are  not  equipped 
with  sufficient  actuators  that  allow  independent  translation 
and  rotation  about  any  given  direction.  In  [2],  the  authors 
proposed  a  controller  design  based  on  feedback  linearization 
technique,  where  the  performance  of  a  hybrid  controller  ca¬ 
pable  of  switching  between  multiple  modes  of  UAV  flight 
(such  as  take-off,  hover,  landing,  and  so  on)  was  demon¬ 
strated.  However,  due  to  the  coupled  nature  of  the  trans¬ 
lational  and  rotational  dynamics  of  a  quad-rotor,  the  most 
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common  control  design  technique  employed,  including  in  this 
paper,  is  the  backstepping  approach. 

In  addition  to  control  design  challenges,  there  exists  the 
problem  of  accurate  measurement  of  position  and  orienta¬ 
tion  in  such  machines.  To  this  end,  vision  sensors  are  being 
increasingly  utilized  within  the  feedback  loop  of  such  sys¬ 
tems.  An  off-board  vision  system  and  a  fixed  camera  were 
used  in  [2]  in  order  to  estimate  the  position  and  orienta¬ 
tion  of  the  UAV  by  tracking  artificially  embedded  visual 
features  on  the  UAV.  In  a  subsequent  work  [3],  the  authors 
also  proposed  the  use  of  multiple  cameras  (a  pan-tilt  unit 
on  the  ground  and  a  camera  on-board,  in  line  of  sight  of 
each  other)  in  an  effort  to  improve  the  accuracy  of  position 
estimation.  Experimental  results  for  different  approaches  to 
utilizing  vision  in  position  sensing  in  helicopters  have  also 
appeared  in  [21,  23].  One  of  the  difficulties  associated  with 
visual  servoing  is  to  ensure  that  the  target  features  remain 
within  the  field  of  view  of  the  camera  at  all  times.  Metni 
et  al.  [20]  proposed  a  controller  with  bounds  imposed  on 
the  UAV  orientation  using  saturation  functions,  in  an  effort 
to  maintain  the  direction  of  the  optical  axis  of  the  camera 
fixed  on  the  UAV  as  close  as  possible  to  the  direction  of  the 
visual  targets,  hence,  keeping  the  targets  within  the  field  of 
view.  Active  control  of  pan-tilt  cameras  have  also  been  pro¬ 
posed  [9,  25]  to  help  maintain  view  of  targets.  Visual  servo 
controllers  usually  fall  into  one  of  the  following  two  classes 
-  image-based  visual  servoing  (IB VS),  and  position-based 
visual  servoing  (PBVS)  [11],  depending  upon  whether  the 
controller  is  designed  to  directly  act  on  the  visual  informa¬ 
tion  (i.e.,  in  image  space),  or  whether  the  visual  information 
is  first  utilized  in  the  estimation  of  the  Euclidean  pose,  re¬ 
spectively.  An  IB  VS  scheme  was  presented  in  [15]  where 
the  dynamics  of  features  in  image  space  were  formulated  in 
terms  of  their  spherical  projections  (a  spherical  image  sur¬ 
face  as  opposed  to  planar).  A  relatively  recent  development 
is  the  homography-based  approach  to  visual  servoing  (also 
called  2^D  visual  servoing  [17]),  which  utilizes  a  combina¬ 
tion  of  partially  reconstructed  Euclidean  information  and  2D 
image-space  information  in  the  control  design,  and  has  been 
shown  to  have  many  practical  advantages.  As  mentioned 
in  [17],  this  method  does  not  require  an  accurate  Euclid¬ 
ean  model  of  the  environment,  and  potential  singularities 
in  the  image  Jacobian  are  avoided.  The  homography  ma¬ 
trix  is  computed  from  two  views  -  a  reference  image  and 
images  captured  online  from  a  moving  camera.  Decomposi¬ 
tion  of  the  homography  matrix  provides  information  about 
the  motion  of  the  camera  between  the  two  views  in  terms  of 
a  rotation  matrix  and  a  translation  vector  that  is  scaled  by 


the  distance  between  the  targets  and  camera  at  the  reference 
position.  Usually,  this  constant  scale  factor  is  unknown  in 
a  visual  servoing  task,  and  hence,  must  be  estimated  online 
if  required  in  the  control  design.  In  [6],  an  adaptive  estima¬ 
tion  technique  was  utilized  to  compensate  for  this  unknown 
depth  information  in  the  visual  servoing  of  a  wheeled  mobile 
robot.  The  idea  presented  in  [6]  was  subsequently  extended 
to  the  problem  of  trajectory  tracking  for  a  UAV  by  other 
researchers  [19]. 

In  a  preliminary  version  of  this  work  presented  in  [7],  we  uti¬ 
lized  the  image  information  from  a  monocular  camera  fixed 
to  the  body  frame  of  a  quad-rotor  UAV  to  develop  a  con¬ 
troller  for  landing  the  UAV  autonomously.  The  position  and 
orientation  errors  were  formulated  by  comparing  the  stream 
of  images  captured  from  the  on-board  camera  to  a  previously 
acquired  reference  image  taken  by  the  camera  when  the  UAV 
was  at  the  desired  final  position  and  orientation  on  the  land¬ 
ing  pad.  Hence,  if  the  images  from  the  camera  match  the 
desired  reference  image,  the  control  objective  will  have  been 
achieved.  However,  due  to  the  underactuated  nature  of  the 
aircraft,  the  orientation  of  the  UAV  cannot  always  be  con¬ 
trolled  to  guarantee  that  both  the  images  will  match,  since 
two  out  of  the  three  rotational  degrees  of  freedom  (d.o.f)  are 
lost  in  achieving  the  position  objective.  In  this  paper,  we 
utilize  a  pan-tilt  camera  on  the  UAV  to  eliminate  this  prob¬ 
lem.  With  the  two  additional  rotational  d.o.f  offered  by  the 
PTCU,  the  combined  UAV-PTCU  system  can  be  controlled 
such  that  the  desired  and  actual  images  from  the  camera 
coincide.  With  this  new  modification  to  the  vision  system, 
we  extend  our  previous  work  towards  the  development  of  a 
vision-based  controller  for  a  UAV  to  autonomously  follow 
a  leader  vehicle  ( e.g .  another  aircraft  or  a  ground  vehicle). 
The  relative  position  and  orientation  between  the  leader  and 
the  camera  frame  on  the  follower  are  obtained  by  comparing 
the  images  captured  by  the  camera  on  the  follower  with  a 
reference  image  of  the  leader  captured  apriori.  The  error 
signals  are  formulated  such  that  the  unknown  scale  factor, 
resulting  from  the  decomposition  of  the  homography  ma¬ 
trix,  is  not  directly  utilized  in  the  control  design.  Similar 
to  the  approach  followed  in  [1]  and  [7],  a  constant  design 
vector  is  integrated  into  the  controller,  resulting  in  an  input 
matrix  that  facilitates  an  advantageous  coupling  of  transla¬ 
tional  dynamics  of  the  UAV  to  the  rotational  torque  inputs. 
In  Appendix  I,  it  is  shown  that  the  leader-follower  strat¬ 
egy  can  also  be  adapted  to  vision-based  trajectory  track¬ 
ing,  where  the  desired  trajectory  of  the  UAV  is  defined  in 
terms  of  a  sequence  of  images  of  stationary  features  on  the 
ground,  captured,  for  example,  during  a  previous  flight  of 
the  UAV.  The  approach  proposed  in  this  paper  has  many 
advantages  relative  to  similar  work  in  the  literature.  Only 
a  single  camera  is  utilized  for  visual  sensing.  The  controller 
on  the  follower  UAV  does  not  require  any  information  from 
the  leader,  such  as  leader  velocity  or  acceleration.  A  model 
describing  the  motion  of  the  leader  vehicle  is  not  required. 
Also,  a  camera  mounted  on  a  pan-tilt  unit  is  more  effective 
in  maintaining  line  of  sight  with  feature  points  on  the  leader 
vehicle  compared  to  a  body  fixed  camera  with  limited  held 
of  view. 


Figure  1:  A  ‘Draganflyer’  quad-rotor  aircraft  with  an  on¬ 
board  pan-tilt  camera. 

2  System  Model 

Our  development  platform  is  a  quad-rotor  helicopter.  The 
vehicle,  shown  in  Figure  1,  is  a  four  rotor  aircraft  capable  of 
vertical  take-off  and  landing  (VTOL).  In  the  figure,  the  pan¬ 
tilt  camera  unit  (PTCU)  is  shown  mounted  on  the  forward- 
looking  limb  of  the  aircraft.  However,  for  the  sake  of  simplic¬ 
ity  in  the  subsequent  development,  we  will  assume  that  the 
optical  center  of  the  camera  coincides  with  the  origin  of  the 
body  fixed  frame  on  the  UAV.  The  camera  is  mounted  such 
that  the  optical  axis  of  the  camera  points  down  along  the  z 
axis  of  the  UAV  body  fixed  frame  when  pan  and  tilt  angles 
are  zero.  We  will  adhere  to  the  following  convention,  bor¬ 
rowed  from  Fossen  [5],  throughout  the  paper:  vector  quan¬ 
tities  denoting  UAV  position  and  velocity  are  specified  as 
v™b,  which  denotes  the  vector  quantity  v  associated  with  the 
b  frame  defined  relative  to  the  e  frame,  and  expressed  in  n 
frame.  We  use  Lx\  V,  and  ‘c T  to  denote  position,  transla¬ 
tional  velocity  and  angular  velocity,  respectively.  Rotation 
matrices  are  specified  as  Rf  G  SO (3)  denotes  a  3  x  3  matrix 
that  transforms  coordinates  defined  in  /  frame  to  t  frame. 

2.1  Dynamic  Model  of  a  UAV 

The  UAV  that  we  are  considering  is  fully  actuated  with  re¬ 
spect  to  orientation  but  underactuated  with  respect  to  trans¬ 
lation  (z.e.,  the  UAV  is  equipped  with  only  one  control  input 
(the  thrust  force)  to  facilitate  translational  motion).  The 
control  development  is  focused  on  the  rigid  body  dynam¬ 
ics  of  the  UAV  (i.e.j  actuator  dynamics  are  not  considered 
within  the  scope  of  the  design).  After  denoting  /  and  F  as 
the  inertial  frame  and  the  follower  UAV  body  fixed  frame, 
respectively,  the  rigid  body  dynamics  are  described  by  the 
following  equations  [10] 
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where  S(-)  G  R3x3  denotes  a  skew-symmetric  matrix  defined 
in  [24],  M  G  R3x3  denotes  the  constant  moment  of  inertia 
around  the  center  of  mass  expressed  in  body  frame  F,  and 
m  G  R1  represents  the  constant  mass  of  the  UAV.  The  term 
Ni  (vfF,  RF,t)  G  R3  represents  the  sum  of  gravitational 


forces  and  additional  time  varying  unmodeled  bounded  dy¬ 
namics  such  as  aerodynamic  resistance.  Similarly,  the  term 
N2  £  R3  includes  unmodeled,  bounded  dis¬ 

turbances  within  the  rotational  dynamics.  The  forces  and 
torques  on  the  rigid  body  due  to  the  actuators  are  denoted 
by  Ff  (t),  Ff  (t)  G  R3,  respectively,  expressed  in  the  body 
frame  F,  and  given  as  follows 

Ff  =  [0  0  U1  f,  (5) 

Ff  —  [  U2  U3  U4  ]T  (6) 

where  Ui(t)  G  R1  denote  the  four  scalar  signals  that  are  used 
to  control  the  UAV. 


F 


Remark  1  The  four  rotor  velocities  vji  G  R1  of  a  quad- 
rotor  UAV  are  related  to  the  rigid  body  forces  Ff  (t)  and 
torques  Ff  (t)  via  the  following  relationship  [10] 
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where  d  G  R1  denotes  the  displacement  of  each  rotor  rela¬ 
tive  to  the  center  of  mass  of  the  airframe,  and  k,b  G  R1 
are  constant  parameters  that  depend  on  the  construction  and 
aerodynamic  properties  of  the  rotor  blades. 


2.2  Kinematics  of  the  Pan-Tilt  Camera  Unit 

The  PTCU  is  mounted  on  the  UAV  such  that  the  camera 
frame,  denoted  by  C ,  coincides  with  the  UAV  frame  F  when 
the  pan  and  tilt  angles,  denoted  by  0p(t),0t(t)  G  R,  respec¬ 
tively,  are  zero.  At  this  configuration,  the  optical  axis  of 
the  camera  points  down  along  the  z  axis  of  the  UAV.  Based 
on  this  geometry,  the  standard  Denavit-Hartenberg  proce¬ 
dure  [24]  can  be  applied  to  compute  the  camera  kinematics 
relative  to  the  UAV  as  follows 
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where  c{0),s(0)  G  R1  represent  the  sine  and  cosine,  respec¬ 
tively,  and  Jc(t)  Gl3x2  denotes  the  Jacobian  for  the  PTCU. 


Figure  2:  Coordinate  frame  assignments  and  geometrical 
relationships. 


UAV  frame  F  coincide;  however,  these  frames  are  shown 
separate  in  the  figure  for  the  sake  of  clarity.  The  leader 
vehicle,  whose  body  fixed  frame  is  denoted  by  L,  is  aug¬ 
mented  with  coplanar  features  whose  image  coordinates  can 
be  easily  tracked  by  processing  the  video  stream  from  the 
camera  mounted  on  the  follower  UAV.  Let  the  constant  vec¬ 
tor  Si  G  R3  expressed  in  the  leader  frame  L  define  the  co¬ 
ordinates  of  the  ith  feature  on  the  leader  vehicle,  and  let 
mft)  =  [  Xi(t)  yft )  zft)  ]T  G  M3  denote  the  Euclid¬ 
ean  coordinates  of  the  same  feature  defined  in  the  follower 
UAV’s  camera  frame  C.  The  frame  L*  in  Figure  2  denotes 
the  leader  at  the  desired  position  and  orientation  relative  to 
the  camera  frame  on  the  follower,  denoted  by  C  R3 

and  Rf*  G  SO( 3),  respectively,  and  m*  G  R3 denotes  the 
constant  Euclidean  coordinates  of  the  ith  feature  point  rel¬ 
ative  to  the  camera  at  this  position.  Based  on  geometry  of 
Figure  2,  the  expressions  for  rrii(t)  and  m*  are  as  follows 


mi  =  XcL+RLSii  (11) 

mi  =  Xcl*  +  R^j*Si.  (12) 

After  solving  (12)  for  si  and  then  substituting  the  resulting 
expression  into  (11),  we  have 

rm  =  x  +  Rm*  (13) 

where  R  (t)  G  SO  (3)  and  x  (t)  G  R3  are  defined  as  follows 

R  =  RclRc\  (14) 

X  =  xcl-Rxcl *•  (15) 


Remark  2  In  the  subsequent  development,  we  will  assume 
that  the  dynamics  of  the  PTCU  can  be  ignored  (i.e.,  the 
pan  and  tilt  angular  velocities,  denoted  by  0c(t),  are  directly 
related  to  the  respective  actuator  inputs  by  proportionality 
constants  only). 


In  Figure  2,  n*  G  R3  denotes  the  constant  normal  to  the 
plane  of  features  on  the  leader  at  desired  relative  position, 
expressed  in  the  camera  frame  C ,  and  the  constant  T  /  0  G 
R1  denotes  the  distance  of  this  plane  from  the  camera.  It 
can  be  seen  from  Figure  2  that  the  projection  of  m*  for  all 
features  along  the  unit  normal  n*  is  given  by 


2.3  Vision  System  Model 

The  geometric  relationships  between  the  leader  and  the  fol¬ 
lower  vehicles  are  shown  in  Figure  2.  As  mentioned  previ¬ 
ously,  the  origins  of  the  camera  frame  C  and  the  follower 


d*  =  n*Tm*.  (16) 

Using  (16),  the  relationship  in  (13)  can  be  written  in  terms 
of  a  Euclidean  homography  H(t)  G  R3x3  [14]  relating  the 


coordinates  nrii(i)  and  m*  as  follows 
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The  measurable  2D  homogeneous  image  coordinates  of  the 
features,  denoted  by  pi(t),p*  £  R3,  expressed  in  the  coordi¬ 
nates  of  frame  C,  are  related  to  the  normalized  Euclidean 
coordinates  and  ,  respectively,  by  the  pin-hole  cam¬ 

era  model  [14]  such  that 


(18) 


where  A  £  R3x3  is  a  known,  constant,  upper  triangular  and 
invertible  intrinsic  camera  calibration  matrix,  and  zi{t),  z*  £ 
R1  are  the  third  coordinate  elements  in  the  vectors  nrii(t)  and 
ra*,  respectively.  Hence,  the  relationship  in  (17)  can  now  be 
expressed  in  terms  of  image  coordinates  of  the  features  as 
follows 


Pi  = 


ai 


a{r  +  tm*Tj  A  1  p* 
\  y 
G 


(19) 


where  ai(t)  £  R1  denotes  the  depth  ratio  and  G(t)  £ 
R3x3  is  a  full  rank  homogeneous  collineation  matrix  [14]. 
Given  pairs  of  correspondences  ( Pi{t),p *)  for  a  minimum  of 
four  coplanar  non-collinear  features  on  the  leader,  the  set  of 
linear  equations  in  (19)  can  be  solved  to  compute  a  unique 
G(t)  up  to  a  scale  factor  [14].  When  more  than  four  fea¬ 
ture  point  correspondences  are  available,  G(t)  can  also  be 
recovered  (again,  up  to  a  scale  factor)  using  techniques  such 
as  least-squares  minimization.  G(t)  can  then  be  used  to 
uniquely  determine  H(t ),  taking  into  account  the  fact  that 
the  intrinsic  camera  calibration  matrix  A  is  assumed  to  be 
known  [14].  By  utilizing  the  algorithms  described  in  [4,  14], 
H  (t)  can  be  decomposed  to  recover  the  rotational  compo¬ 
nent  R(t)  and  the  scaled  translational  component  -^x{t). 
The  signals  R(t )  and  A- x(t )  are  the  measurable  quantities 
from  the  vision  system  that  we  will  exploit  to  design  a  con¬ 
troller  for  the  follower  UAV. 


Remark  3  If  all  the  features  on  the  leading  vehicle  are  not 
coplanar,  the  collineation  matrix  in  (19)  can  still  be  devel¬ 
oped  by  employing  the  virtual  parallax  algorithm  described 
in  [17].  However,  in  this  case,  at  least  eight  non- coplanar 
features  must  be  utilized. 


Remark  4  Decomposition  of  the  Homography  matrix  H(t) 
results  in  four  unique  solutions,  two  of  which  are  physically 
impossible.  From  the  remaining  two  solutions,  the  right  one 
can  be  selected  by  utilizing  some  additional  information,  such 
as  a  guess  for  the  normal  vector  n* . 

3  Control  Formulation 

The  control  objective  is  to  ensure  that  the  follower  UAV  and 
the  PTCU  are  controlled  such  that  the  images  of  the  features 


on  the  leader,  seen  by  the  camera  on  the  follower,  matches 
the  reference  image  of  the  same  features  captured  by  the 
camera  when  the  leader  is  at  a  desired  relative  position  and 
orientation.  Based  on  Figure  2  and  the  development  in  the 
previous  section,  this  can  be  stated  as  the  desire  to  control 
the  follower  UAV  such  that  the  following  control  objective 
is  achieved 


%cl  ~ ^  xcl*iRl  Rl*  as  t  — y  oo.  (20) 

An  examination  of  equations  (14)  and  (15)  reveals  that  the 
control  objective  stated  above  can  be  achieved  if  the  follower 
UAV  and  the  PTCU  can  be  controlled  such  that  R{t)  hx3 
and  x(t)  — y  0  as  t  — y  oo,  where  the  notation  Inxn  denotes 
an  n  x  n  identity  matrix.  To  achieve  the  stated  objective, 
we  make  the  following  assumptions 

Assumption  1:  The  velocities  vfF(t)  and  ovfF{t)  are  mea¬ 
surable  from  inertial  sensors  on-board  the  follower 
UAV. 

Assumption  2:  The  uncertain  nonlinearties  in  the  UAV 
translational  and  rotational  dynamics,  denoted  by 
TVi(-)  and  A^-)  in  (2)  and  (4),  respectively,  can  be 


upperbounded  in 

the  following  manner 

IIJVill 

< 

^  (1 

1  F 
|  Wf 

1) 

(21) 

ft: 

to 

< 

Ml 

1  F 

|  Wf 

1  ’  IK!) 

(22) 

where  t/q(-)  £  R1  are  positive,  scalar,  non-decreasing 
functions  of  their  arguments. 

Assumption  3:  The  pan  and  tilt  angles  of  the  camera  are 
measurable  from  the  resolvers  in  the  PTCU.  We  also 
assume  that  the  dynamics  of  the  PTCU  can  be  ig¬ 
nored,  i.e .,  the  angular  velocities  of  the  pan  and  tilt 
angles  of  the  PTCU  can  be  directly  controlled. 

Assumption  4:  The  pan  angle  of  the  camera  is  confined 
to  the  region  <  Qp(t)  < 

Assumption  5:  The  scaled  position  and  orientation  vari¬ 
ables  -^x(t)  and  R(t)  are  available  from  the  vision 
system. 

Assumption  6:  The  position  and  velocities  of  the  leader 
vehicle  are  bounded,  i.e.,  xIIL{t),vjL{t),ujIIL{t)  £  Coo- 
However,  these  signals  are  not  available  to  the  con¬ 
troller  on  the  follower  UAV,  i.e.,  there  is  no  commu¬ 
nication  between  the  vehicles. 

Assumption  7:  The  features  on  the  leader  are  in  front  of 
the  camera  on  the  follower  {i.e.,  Zi  >7  for  every  fea¬ 
ture  point,  where  7  £  R1,  is  a  positive  scalar  con¬ 
stant),  and  the  features  on  the  leader  remain  within 
the  field  of  view  of  the  camera  on  the  follower. 

3.1  Orientation  Error  Formulation 

The  axis-angle  representation  [24]  of  the  rotation  matrix 

R{t)  defined  in  (14)  is  utilized  to  define  the  orientation  error, 

denoted  by  ee{t)  £  R3,  as  follows 

ee  =  ix<f> 


(23) 


where  n(t)  E  R3  represents  a  unit  axis  of  rotation,  and  (f(t)  E 
R1  denotes  the  rotation  angle  about  fi(t)  (confined  to  the 
region  —7 r  <  <  7r),  explicitly  dehned  as  follows 

^  =  =  ^  (24) 


where  the  notation  tr(-)  denotes  the  trace  of  a  matrix.  After 
taking  the  time  derivative  of  the  orientation  error  system 
in  (23),  and  utilizing  the  PTCU  kinematics  from  (9),  the 
open  loop  dynamics  for  the  orientation  error  system  can  be 
expressed  as  follows 

T  C 

eg  —  L^locl 

—  — 1  f  —  LujRpJcOc  T  LuUJjl  (25) 

where  L^{t)  E  R3x3  is  a  Jacobian-like  matrix  given  by  the 
following  expression 


Lw  =  hx3  -  | S»  + 
SmcW^. 


sine  ( <j> ) 


\ 

/ 


5(p)2, 


(26) 


The  development  of  the  expression  for  (£)  is  presented  in 
detail  in  [8]  and  [16],  and  utilizes  the  time  derivative  of  (14), 
which  provides  an  expression  for  the  time  derivative  of  R(t ) 
as  follows 

R=  S  (wgL)  R  (27) 

where  the  following  property  was  utilized  in  its  derivation 

Rl  =  S  (wgL)  Rcl.  (28) 


Remark  5  By  exploiting  the  fact  that  fi(t)  is  a  unit  vector 
(i.e.j  \\n\\2  =  1),  the  determinant  of  (t)  can  be  derived  as 
follows  [16] 


det  {Lu} 


(29) 


From  (29),  it  can  be  seen  that  I/J1(t)  is  only  singular  for 
multiples  of  2tt,  which  is  outside  the  assumed  range  for  (f(t) . 
Lu(t)  is  assumed  invertible  in  the  subsequent  development. 


3.2  Position  Error  Formulation 

The  position  error  is  denoted  by  Xh(t)  G  R3,  and  dehned  as 
follows 

xh  =  fx.  (30) 

The  open  loop  dynamics  of  the  position  error  system  can  be 
obtained  by  taking  the  time  derivative  of  (15)  as  follows 

d*  xh=  xcl~  R  xcl*  •  (31) 


From  the  geometry  in  Figure  2,  it  can  be  seen  that 

xcl  —  R^Rf  (xil  —  xIF^j  (32) 

and  the  time  derivative  of  (32),  after  simplification,  can  be 
expressed  as  follows 


•  C  tdC  f  0  f  c  \  c  .  c 
Xcl  —  —RfvIF  —  S  [f^IC  J  XCL  +  VIL 


where  (3)  and  (10)  have  been  utilized.  After  substituting 
(33)  in  (31),  and  utilizing  (27)  for  the  time  derivative  of 
R(t),  the  following  expression  for  the  open  loop  position 
error  dynamics  can  be  obtained 

d *  Xh  —  —Rfv[f  —  S  d*Xh 

—S  RxCcl*  +  vfL.  (34) 

Based  on  the  subsequent  Lyapunov  stability  analysis,  an 
auxiliary  signal  r(t)  E  R6  is  dehned  as  follows 

r  =  [  rj  ej  ]T  (35) 


where  ee(t)  is  the  orientation  error  dehned  in  (23),  and 
rp(t)  E  R3  is  dehned  in  the  following  manner 

rp  —  kpXh  —  Rp'Vjp1  T  RfS  (36) 

where  kp  E  R1  denotes  a  positive,  scalar  constant,  and 
6  =  [  Si  S 2  S3  J  E  R  represents  a  constant  design  vec¬ 
tor.  The  constant  5  introduced  in  (36)  facilitates  an  advan¬ 
tageous  coupling  between  the  translational  and  rotational 
dynamics  of  the  UAV.  Upon  substituting  (36)  in  (34),  the 
following  alternate  expression  for  the  open  loop  position  er¬ 
ror  dynamics  is  obtained 


d*  Xh  =  rp  —  kpXh  -  RfS  -  S  (v?c)  d* Xh 

—S  Rxcl*  (37) 

After  taking  the  time  derivative  of  (35),  and  substituting 
(34)  for  Xh  (t),  the  dynamics  in  (2)  for  hfF(t),  (10)  for  Rp  (t), 
and  (25)  for  ee(t),  the  following  expression  for  the  open  loop 
dynamics  of  r(t)  can  be  developed  after  some  mathematical 
manipulation  and  simpliheation 


r 


—S  (cafe)  rp 

03xi 


+ 


T  N11  +  N\2 


RcfS{^f)8-±RcfF? 
-LuR%ufF  -  FRpJJ 


c 


(38) 


where  0 „xm  denotes  a  n  x  m  zero  matrix,  and 
iVii(t),  iVi2(i)  €  R6  groups  the  following  terms  in  (38) 


Nn  = 

N±2  4 


'  -^RfvFf  -  ±RGfNi  ' 

_  Oaxi  J  ’ 


'  -%S(u?l)Rx ccl,  +  ^l  - 
Lujcoff 


(39) 

(40) 


The  second  term  in  (38),  after  re-arrangement,  can  be  writ¬ 
ten  as  the  product  of  a  matrix  B(t)  E  R6x6  and  a  vector 
U(t)  E  R6,  where 

B  =  B±B2, 

73  _  —R^  03x3 

1  W  [  03X3  —LuRp  \  5 


(41) 

(42) 
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(43) 


u 


u  1 


Ll>iF 
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(44) 


(33) 


Hence,  from  (38),  and  (41)  through  (44),  it  can  be  shown 
that 

r  =  \  S  F&9  rp  ]  +BU  +  Nu  +  N12.  (45) 

d3xi 

Remark  6  The  matrix  B2(t)  is  invertible  if  S3  7^  0  and  at 
least  one  of  Si  or  82  is  non-zero.  Since  B\(t)  is  invertible, 
it  follows  that  B(t)  is  invertible  if  S  is  selected  to  satisfy  the 
above  mentioned  constraints. 

4  Control  Design 

The  control  design  is  based  on  the  backstepping  technique. 
An  auxiliary  control  signal  Un  (t)  G  R6  and  an  angular  veloc¬ 
ity  backstepping  error  signal  77 (t)  G  R3  are  defined  as  follows 


where  (L(-)  G  R1  are  positive,  scalar,  non-decreasing  func¬ 
tions  of  their  arguments.  The  subsequent  control  design 
requires  that  Ci(')  be  differentiable,  and  hence,  it  is  con¬ 
structed  as  a  function  of  a  modified  norm,  denoted  by  ||-||s, 
which  is  defined  in  the  following  manner 

II2/IL  -  VyTy  +  a,  Vy  e  R3  (56) 

where  a  G  R1  represents  a  small  positive  constant.  Based 
on  the  stability  analysis  presented  in  the  next  sub-section, 
the  control  inputs  U (t)  and  F 'F  (t)  are  designed  as  follows 

U  =  —krr  —  Xh  -r— ,  (57) 

03xl  J  £l 

FtF  =  MUUnl  +  S  (ufF^  Mu;?f 


jr  hT  1 

(46) 

r.  lT 

Uif 

(47) 

where  cjn(t)  G  R3  represents  a  desired  angular  velocity  sig¬ 
nal.  From  (44),  (46)  and  (47),  the  following  relationship  can 
be  observed 

U  =  Un-  UTr]  (48) 

where  n  G  R3x6  denotes  the  following  constant  matrix 

"  0  1  0  0  0  0  " 

n  =  0  0  1  0  0  0  .  (49) 

_  0  0  0  1  0  0  _ 

The  auxiliary  control  input  Un  ( t )  is  designed  in  the  following 
manner 

Un  =  B~XU  (50) 

where  U(t)  €  R6  is  a  subsequently  designed  control  input. 
From  (45),  (48)  and  (50),  the  open  loop  dynamics  for  r(t) 
can  now  be  expressed  as  follows 

r  =  \  S  rp  ]  +  U  +  BUtV  +  Nu  +  N12.  (51) 

U3x 1 

Similarly,  after  differentiating  (47)  and  substituting  for 
ujfF(t)  from  the  UAV  rotational  dynamics  given  by  (4),  the 
open  loop  dynamics  for  the  angular  velocity  error  signal  77 (t) 
are  obtained  as  follows 


C2 

+kvr)  -  n BTr  +  ^  (58) 

£2 

where  kr,kv  G  M1  are  positive,  scalar  control  gains,  £1,^2  G 
R1  are  positive,  scalar  constants. 


Remark  7  The  function  in  (56)  is  utilized  in  (54)  instead 
of  the  standard  Euclidean  norm  to  ensure  that  the  time  deriv¬ 
ative  of  C i('),  which  appears  in  Uni{t),  is  well-defined.  The 
time  derivative  of  ||-||s  is  expressed  as  follows 


4.1  Stability  Analysis 

Theorem  1  Given  the  error  dynamics  of  (37),  (51)  and 
(52),  the  translational  force  input  and  the  rotational  torque 
input  developed  in  (57)  and  (58),  respectively,  guarantees 
uniform  ultimate  boundededness  (UUB)  in  the  position  er¬ 
ror  signal  Xh(t )  and  the  orientation  error  signal  e$(t)  in  a 
neighborhood  about  zero,  in  the  sense  that 

||*h(t)|| ,  ||ee(£)||  <  Qi  exp  (~a2t)  +  a3  (60) 

where  01,02,0:3  €  R1  are  adjustable,  positive  constants. 


Mr, 1  =  MUUm  +  S  (wfF)  MuF1F  -  N22  -  F f  (52) 

where  the  measurable  and  unmeasurable  terms  have  been 
grouped  into  Un\(t)  G  R3  and  Un2(t)  G  R3,  respectively 
(See  Appendix  II  for  explicit  derivations),  and  finally,  all 
unmeasurable  terms  in  the  rotational  dynamics  have  been 
combined  into  a  single  term  N22(t)  G  R3  defined  in  the  fol¬ 
lowing  manner 

N22  =  N2-  MUUn2.  (53) 


Proof:  We  choose  the  following  non-negative  scalar  function 
as  the  Lyapunov  candidate  to  prove  the  above  theorem 

V  4  ^dTxlxh  +  \rTr  +  \r,T MV.  (61) 

After  taking  the  time  derivative  of  (61),  substituting  the 
dynamics  for  xh  (t),r(t)  and  fj(t)  from  (37),  (51)  and  (52), 
and  substituting  the  expressions  for  ?7(t)and  Ff  (t)  from 
(57)  and  (58),  the  following  expression  can  be  obtained 


From  Assumption  2,  (39)  and  (53),  it  is  easy  to  see  that  the 
uncertain  terms  Nn(t)  and  N22(t)  can  be  upperbounded  in 
the  following  manner 


V  <  -kp  1  lla^ll2 


1 1  -Vi  1 1 1  <  Ci  (||v/f||  )  (54)  +  (ll^hll  \Pi 

;  p  I,  p  +(\\rm\- 

i JV22 11  <  C2  ’  |a,jFl)  (55) 
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-kr,  ||l?H 
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(62) 


where,  based  on  Assumption  4,  /31?  f32  G  M1  are  two  positive, 
bounding  constants  chosen  such  that 

Pi  >  ||^||  +  ||5(^fL)  i^gL*||  +  ||^||,  (63) 

P2  >  ||7V12||  (64) 

and  kpi,  kr i,  kp 2,  /cr2  G  R1  are  positive,  scalar  constants  such 
that  kp  =  kpi  +  kP2  and  kr  =  kr  1  +  kr2 -  After  applying  the 
non-linear  damping  argument  from  [12],  the  bracketed  terms 
in  the  above  expression  can  be  upper-bounded  as  follows 

IMICi  (i  -  —y)  < 

NIC2(i-^)  < 

WxhHlPil-k^WxhW)  < 

Wd^l-fcraW)  < 

From  (62)  and  (65),  V(t)  can  be  upper  bounded  in  the  fol¬ 
lowing  manner 

V  <  -kp  1  ||x^||2  -  Ki  \\r\\2  -  kv  \\rj\\2  +  e  (66) 

where  e  =  £i+£2  +  +  G  R1.  The  solution  to  the  above 

Kp2  Kr 2 

differential  equation  allows  us  to  upper-bound  the  tracking 
errors  in  the  following  manner 


ei, 


£2, 

\M 

kp2  ’ 

\M 

kr2  ' 


(65) 


\\xh(t)\\ ,  \\ee(t)\\  <V(t)<  ai  exp  (- a2t )  +  a3  (67) 

,  T//nx  min(fcpi,fcri,fcn) 

where  ai  =  F(0),  <a2  —  - (\  - " — w,  and  a3 

max  f  — ,  d*,  Amax  (M)  J 

—  are  positive  scalar  constants,  and  Amax(M)  G  R1  denotes 
<a2 

the  largest  eigenvalue  of  the  inertia  matrix  of  the  follower 


UAV. 


From  (61)  and  (66),  it  can  be  seen  that  Xh(t),r(t),r](t)  G 
Coo.  From  (35)  and  (36),  it  follows  that  vfF(t),eo(t)  G  £00, 
and  consequently,  from  the  UAV  translational  dynamics  in 
(i  ),±fF(t)e  Coo  -  Since  Xh(t)  is  bounded,  from  (15)  and  Fig¬ 
ure  2,  it  can  be  seen  that  XjF(t)  G  Coo-  From  (39),  JVn(-)  G 
Coo ,  and  hence,  from  (54),  Ci(m)  C  Coo-  It  follows  from 
(57)  and  (54)  that  U(t)  G  Coo-  Consequently,  it  is  easy  to 
show  that  Un(t),  U(t)  G  Coo  from  (50)  and  (48),  respectively; 
therefore,  ujn(t),ufF(t),dc(t),ui(t)  G  Coo-  The  PTCU  Ja¬ 
cobian  Jc(t)  is  a  bounded  function  of  pan/tilt  angles  0c(t) 
and  hence,  from  (9)  it  can  be  inferred  that  u>FC(t)  G  Coo, 
which  leads  us  to  the  observation  that  ujjc(t)  G  Coo-  We 
have  now  demonstrated  that  every  term  in  the  expressions 
for  the  time  derivatives  of  r(t),  Xh(t)  and  e$(t),  as  shown  in 
(45),  (37)  and  (25),  are  bounded.  As  a  result,  U(t),Un(t) 
G  Coo  (see  (57)),  and  the  follower  translational  acceleration 
vfF(t)  C  Coo  (as  revealed  by  taking  the  time  derivative  of 
Tp{t)  in  (36)).  Hence,  from  (53)  and  (55),  iV22(-),  £2(-)  G  Coo- 
From  the  preceding  signal  chase,  and  from  (58),  we  can  con¬ 
clude  that  Ff  (t)  G  Coo,  and  hence,  from  (4),  c ofF(t)  G  Coo- 
Therefore,  all  signals  remain  bounded  during  closed  loop  op¬ 
eration.  ■ 


Remark  8  The  control  objective  is  formulated  as  the  desire 
to  control  the  follower  UAV  such  that  the  images  obtained 


from  the  PTCU  and  a  reference  image  of  the  features  on 
the  leader  vehicle  coincide.  Since  the  origins  of  the  follower 
UAV  frame  and  the  camera  frame  coincide,  it  can  be  seen 
from  (20)  and  (67)  that  the  controller  developed  in  Section 
4  will  maintain  the  follower  at  a  fixed  position  behind  the 
leader.  However,  the  orientation  objective  in  (20)  is  formu¬ 
lated  as  the  desire  to  control  the  PTCU  orientation  R^it) 
relative  to  the  leader,  and  hence,  we  cannot  state  how  the 
orientation  of  the  UAV  (RF(t))  will  evolve  with  time. 


5  Simulation  Results 


The  following  UAV  parameters  were  used  for  simulation  of 
the  follower  control  algorithm 


m 

M 


0.6  (kg), 

"  0.4  0  0 

0  0.4  0 

0  0  0.6 


(68) 

(kg-m2).  (69) 


The  simulated  leader  vehicle  is  a  mobile  robot  moving  in 
a  circular  trajectory  on  the  ground  plane  (XY).  The  per¬ 
spective  camera  on  the  follower  was  simulated  to  track  eight 
coplanar  feature  points  that  formed  the  corners  of  two  con¬ 
centric  squares  on  top  of  the  leader  vehicle.  In  a  practi¬ 
cal  implementation,  a  real-time  implementation  of  an  auto¬ 
matic  feature  tracking  algorithm  such  as  the  Lucas-Kanade 
tracker  [13]  could  be  utilized  to  track  features  on  the  leader. 
The  homography  matrix  was  computed  from  the  image  co¬ 
ordinates  of  the  features  relative  to  the  camera  frame  using 
the  least-squares  technique.  An  implementation  of  the  algo¬ 
rithm  given  in  [4]  was  utilized  to  decompose  the  homography 
matrix  in  order  to  compute  the  error  signals  given  in  (23) 
and  (30).  The  initial  position  of  the  UAV  was  set  to  5  meters 
above  the  leader  vehicle  (i.e.,  z  —  —5  m),  and  the  desired 
time  varying  position  of  the  follower  UAV  was  selected  to  be 
10  meters  above  the  leader  vehicle.  The  initial  and  desired 
orientation  of  the  UAV  were  [  0.01  0.01  1.57  ]  radians 

and  [  0  0  0  ]T  radians,  respectively.  The  trajectories 

of  the  leader  and  follower  vehicles  are  shown  in  Figure  3, 
while  the  time  evolution  of  position  and  orientation  error 
signals  are  shown  in  Figure  4.  The  UAV  control  effort  and 
the  PTCU  angular  velocity  input  remains  bounded  within 
reasonable  values,  as  shown  in  Figures  5  and  6  respectively. 
For  a  realistic  simulation,  the  UAV  thrust  force  u\(t)  was 
saturated  at  20  N  and  torques  u2{t)  to  u±(t)  were  saturated 
at  5  N-m  to  restrict  initial  spikes  in  the  control  signal.  The 
following  control  gains  were  used,  selected  based  on  trial  and 
error 


kp  =  20,  kr  =  10,  kp  =  100,  (70) 

ei  =  1,  £2  =  0.1,  (71) 

<5  =  [0.1  0  0.5  ]T.  (72) 

The  nonlinearities  iVi(-)  and  7V2(-)  in  (2)  and  (4)  were  as¬ 
sumed  to  be  of  the  following  form 

Ni  =  —  0.5Tanh(irfF)  (73) 

N2  =  -0.5Tanh(wfF)  (74) 


where  Tanh(^)  G  Mn  is  a  vector  whose  elements  are  defined 
as  the  hyperbolic  tangent  of  elements  of  the  vector  v(t)  G 


UAV  trajectory 


Figure  3:  The  leader  (□)  and  follower  (A)  trajectories. 

Rn.  The  scalar  functions  Ci(’)  and  C2O  in  (54)  and  (55), 
respectively,  were  selected  as  follows 

Ci  —  0-1  \\vFf  II  5  (75) 

II  Ms 

C2  —  0-1  1 1  i  f  1 1  T  0-1  II^ifII  •  (76) 


position  error  (x)  position  error  (y)  position  error  (z) 


Figure  4:  Follower  position  errors  ( Xh(t ),  dimensionless). 

Orientation  errors  ( ee(t ),  m-rad.)  were  practi¬ 
cally  zero. 


6  Conclusions 

This  paper  presented  the  design  of  a  vision-based  controller 
for  an  under  actuated  UAV  equipped  with  a  pan-tilt  cam¬ 
era  to  achieve  the  objective  of  following  a  leader  vehicle  au¬ 
tonomously.  The  development  of  the  relative  position  and 
orientation  error  signals  employed  homography-based  tech¬ 
niques,  and  the  controller  design  was  developed  using  the 
backstepping  approach.  The  proposed  controller  was  shown 
to  achieve  uniform  ultimate  bounded  (UUB)  tracking.  Six 
control  inputs  (ui(t)  to  1x4 (t)  and  0c(t))  were  utilized  to  con¬ 
trol  the  six  d.o.f  combined  UAV-PTCU  system  to  achieve 
the  objective  of  matching  the  images  captured  by  the  cam¬ 
era  with  a  reference  image  of  the  leader.  This  would  not  be 
possible  without  the  two  additional  d.o.f.  provided  by  the 
PTCU  to  compensate  for  the  two  rotational  d.o.f.  in  the 
UAV  lost  towards  achieving  the  position  control  objective. 
It  should  be  noted  that  if  the  PTCU  is  removed,  we  can  fol¬ 
low  the  approach  outlined  in  our  previous  work  [7]  to  design 
a  controller  that  achieves  3  d.o.f  position  tracking  and  yaw 
(rotation  about  UAV  vertical  axis)  tracking. 
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Appendix  I:  Extension  to  Vision-Based 
Trajectory  Tracking 

With  a  few  modifications  in  geometric  modeling,  the  leader- 
follower  strategy  described  in  the  paper  can  be  adapted  to  a 
vision-based  trajectory  tracking  strategy,  where  the  desired 
trajectory  of  the  UAV  is  defined  in  terms  of  a  sequence  of 
images  of  stationary  features  on  the  ground,  captured,  for 
example,  during  a  previous  flight  of  the  UAV.  Along  this 
line  of  reasoning,  we  follow  an  approach  similar  to  that  pre¬ 
sented  in  [6].  At  every  instant  of  time,  the  image  captured 
from  the  UAV  in  flight,  and  a  corresponding  image  from  the 
previously  obtained  desired  video  sequence  are  compared  to 
a  reference  image  of  the  ground  features  in  order  to  com¬ 
pute  the  relative  position  and  orientation  tracking  errors. 
The  geometrical  relationships  between  the  various  coordi¬ 
nate  frames  are  shown  in  Figure  7.  As  before,  the  origin 
of  the  camera  frame  is  assumed  to  coincide  with  the  UAV 


Figure  7:  Geometrical  relationships  for  vision-based  tra¬ 
jectory  tracking. 

frame,  and  they  are  shown  separate  from  each  other  in  Fig¬ 
ure  7  for  the  sake  of  clarity.  The  optical  axis  of  the  PTCU  is 
pointed  downwards  by  default,  along  the  z  axis  of  the  cam¬ 
era  frame  and  the  UAV.  The  PTCU  kinematics  are  given 
in  expressions  (8)  to  (10).  Let  the  constant  vector  Si  £  R3 
expressed  in  the  inertial  frame  I  define  the  coordinates  of 
the  ith  visual  feature  on  the  ground,  and  let  rrii{t)  £  R3  de¬ 
note  the  Euclidean  coordinates  of  the  same  feature  defined  in 
the  UAV’s  camera  frame  C.  The  objective  is  to  control  the 
UAV,  denoted  by  coordinate  frame  F,  such  that  the  images 
of  the  visual  features  on  the  ground  obtained  from  the  cam¬ 
era  on  the  UAV  (denoted  by  (7),  coincide  with  the  images  of 
the  same  visual  features  captured  by  the  camera  when  the 
UAV  is  at  the  desired  location,  denoted  by  the  coordinate 
frame  Fd  shown  in  Figure  7.  Let  rriid(t)  £  R3  denote  the 
3D  Euclidean  coordinates  of  the  ith  visual  feature  as  seen 
by  the  camera  on  the  UAV  at  the  desired  position,  and  let 
m*  £  R3 denote  the  constant  Euclidean  coordinates  of  the 
same  feature  point  relative  to  the  camera  frame  when  the 
UAV  is  at  its  reference  position  F* .  Based  on  the  geometry 
in  Figure  7,  the  Euclidean  coordinates  of  the  same  visual  fea¬ 
ture,  as  seen  by  the  on-board  camera  from  the  three  camera 
locations  denoted  by  (7,  Cd  and  (7*  are  related  as  follows 

mi  =  x  +  Rrn *  ,  . 

mid  =  xd  +  Ram * 

where  the  new  position  and  orientation  variables  are  defined 


in  the  following 

manner 

R 

= 

itFitj  rCF*  -tic*  , 

(78) 

Rd 

= 

R%R?dRF*R&, 

(79) 

X 

= 

RFRf  ^xTIF*  —  5 

(80) 

Xd 

= 

RFdRfd  [x\F*  -  x\Fd^j  . 

(81) 

After  defining 

the 

relationship  between  the 

normalized 

Euclidean  coordinates  and  the  2D  homogeneous  image  co¬ 
ordinates  of  the  visual  features  as  shown  in  (18),  the  rela¬ 
tionships  in  (77)  can  be  expressed  in  terms  of  their  image 
coordinates  as  follows 

Pi  =  a, A  ( R+  A~1p*, 

V  d  '  (82) 


where  aid(t)  =  Pidit)  =  €  R1,  n*  £  R3 

denotes  the  normal  vector  to  the  ground  plane  defined  in 
the  frame  (7*,  and  d*  £  R1  is  the  distance  to  the  ground 
plane  from  the  frame  (7*.  If  four  or  more  visual  features  on 
the  ground  can  be  continuously  tracked  by  the  on-board  vi¬ 
sion  system  on  the  UAV,  the  homography  matrices  shown  in 
brackets  in  (82)  can  be  computed,  which  can  be  subsequently 
decomposed  utilizing  the  algorithms  previously  mentioned  in 
order  to  calculate  the  rotation  matrices  R(t)  and  Rd(t ),  and 
the  scaled  translation  vectors  and  Xd^ . 

Control  Formulation:  In  addition  to  Assumptions  1  to  4 
stated  for  the  leader-follower  case,  we  utilize  the  following 
additional  assumptions  in  the  control  development 

Assumption  5:  The  scaled  position  and  orientation  vari¬ 
ables  R(t),  Rd(t),  ^jrx(t)  and  j^Xd(t)  are  available 
from  the  vision  system. 

Assumption  6:  The  desired  position  and  velocity  signals 
are  bounded,  z.e.,  x\Fd(t),v\Fd(t),u\Fd(t)  £  Coo- 
Assumption  7:  All  the  visual  features  tracked  by  the  vi¬ 
sion  system  are  always  within  the  held  of  view  of  the 
camera,  and  are  in  front  of  the  camera  (i.e.,  Zi  >  7  for 
every  feature  point,  where  7  £  R1,  is  a  positive  scalar 
constant). 

The  position  error  signal,  denoted  by  ep(t)  £  R3,  is  defined 
in  the  following  manner 

ep  =  —  ( xif  ~  •  (83) 

After  some  mathematical  manipulation,  it  can  be  shown  that 
the  position  error  signal  defined  in  (83)  can  be  written  in 
terms  of  measurable  signals  from  the  vision  system  as  follows 

ep  =  RcReXdh  -  RcXh  (84) 

where 

Re  =  RRd ,  (85) 

Xdh  =  (86) 

xh  =  ^x.  (87) 

Analogous  to  the  leader-follower  case,  the  objective  of  ori¬ 
entation  tracking  is  achieved  if  Re(t)  13x3  as  t  00; 
hence,  the  orientation  error  signal,  denoted  by  ee{t)  £  R3, 
is  defined  in  terms  of  the  axis-angle  representation  of  the 
rotation  matrix  Re(t)  as  follows 

ee  =  Ijl4>  (88) 

where 

*-  (i  (•,(«•>-»)•  SM'lSif  '  (89) 

After  taking  the  time  derivative  of  the  error  signals  in  (83) 
and  (88),  the  following  open-loop  error  dynamics  are  ob¬ 
tained 

d  ep  —  VjF  —  S  d  ep  —  Rj  XjFd  (90) 

T  C 

GO  —  -L/ijjLU  (j  dc 

=  —LuRfvTf  ~  LuRpJcbc  +  LuUj?Cd-  (91) 


Pid  —  &idA  (  Rd  -{-  ^  Xd,Tl 


A~xp* 


Based  on  the  subsequent  Lyapunov  analysis,  an  auxiliary  in  (46),  (47),  and  (50),  respectively,  the  dynamics  of  r(t)  can 
signal  r(t)  E  R6  is  defined  as  follows  be  expressed  as  follows 


(92) 


where  rp(t)  E  R3  is  defined  in  the  following  manner 

r  p  =  kpep  +  VjF  +  5  (93) 

and  the  constants  kp  E  R1  and  5  E  R3  have  the  same  defi¬ 
nitions  as  in  the  leader-follower  case.  After  substituting  for 
vfF(t)  from  (93)  in  (83),  the  following  alternate  expression 
for  the  open-loop  position  error  dynamics  can  be  obtained 


d*ev 


V p  kpGp  5  $  I  Fj  d  £p 

—RfxiFd-  (94) 


The  time  derivative  of  (92)  is  complex,  but  it  can  be  shown 
that,  after  substituting  (90)  for  ep(t),  the  dynamics  in  (2) 
for  vfF  (t) ,  (10)  for  RF(t),  and  (91)  for  eo(t),  the  following 
simplihed  expression  for  the  open  loop  dynamics  of  r(t)  can 
be  developed 


r  — 


—S  (ufF)  rp 

03X1 


+  BU  +  Nn  +  N12 


where 


(95) 


r  = 


—S  (ujfF)  rp 
03x1 


+  U  -  BUT7]  +  Ni%  +  N12.  (102) 


The  dynamics  for  r/(t)  is  the  same  as  before,  and  given  by  the 
expression  in  (52).  Based  on  the  stability  analysis  provided 
in  the  next  sub-section,  the  control  inputs  are  designed  in 
the  following  manner 


U 

FtF 


—krr  — 


O3  x 


.c? 

5 

£l 


MUUnl  +  S 


(u?f) 


Mujf1f 


_  C2 


+kvr)-HB  r  +  T]  — 
£2 


(103) 


(104) 


where  kr,kv  E  R1  are  positive,  scalar  control  gains,  £1,52  E 
R1  are  positive,  scalar  constants,  and  Ci (*) 5  C2 (*)  £  are 
known  positive,  scalar,  non-decreasing  bounding  functions 
on  ATii (•)  and  N22(-),  constructed  like  (54)  and  (55),  respec¬ 
tively.  Un\{t),  Un2(t)  E  R6  are  the  known  and  unknown 
terms,  respectively,  in  the  time  derivative  of  Un(t ),  and  their 
expressions  are  given  in  Appendix  II. 
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Remark  9  After  utilizing  Assumption  it  can  be  shown 
that  B2{t)  E  R6x6  is  full-ranked  if  63  7^  0.  Since  it  can  be 
easily  verified  that  B\(t)  E  R6x6  is  invertible,  it  follows  from 
(96)  that  B(t)  is  invertible. 


Remark  10  Note  that  the  dynamics  for  ee(t),  ep{t)  and 
r(t),  as  given  by  the  expressions  in  (91),  (94)  and  (95),  re¬ 
spectively,  are  similar  in  form  to  the  dynamics  of  the  error 
systems  for  the  leader-follower  case,  given  by  (25),  (37)  and 
(45).  Hence,  we  follow  the  same  approach  as  the  leader- 
follower  problem  in  the  subsequent  control  design. 


Control  Design:  After  defining  the  desired  control  sig¬ 
nal  tJn(t)  and  the  backstepping  error  signal  f](t),  and  subse¬ 
quently  designing  the  desired  control  signal  Un{t)  as  shown 


Theorem  2  Given  the  error  dynamics  of  (94),  (102)  and 
(52),  the  translational  force  input  and  the  rotational  torque 
input  developed  in  (103)  and  (104),  respectively,  guarantees 
uniform  ultimate  boundededness  (UUB)  in  the  position  er¬ 
ror  signal  ep(t)  and  the  orientation  error  signal  eo(t)  in  a 
neighborhood  about  zero  in  the  sense  that 

HepOOII  5  lle0 (OH  <  OLi  exp  ( -a2t )  +  o3  (105) 

where  aq,  02,0^3  E  R1  are  adjustable,  positive  constants. 


Proof :  We  choose  the  following  non-negative  scalar  function 
as  the  Lyapunov  candidate  to  prove  the  above  theorem 

V  =  ejep  +  trTr  +  Mr) .  (106) 

After  taking  the  time  derivative  of  (106),  substituting  the 
dynamics  for  ep(t),r(t)  and  rj(t)  from  (94),  (102)  and  (52), 
and  substituting  the  expressions  for  U (t)  and  Ff  (t)  from 
(103)  and  (104),  the  following  expression  can  be  obtained 


V  < 


-kp i  ||ep||2  -  kri  ||r||2 

/  IUII2/"2\ 


V"  " ) 

+  (llep|l  \Pi\  —  kp 2  ||ep|| 
+  (IMI  \P2\ -  kr2  ||r||2) 


~kv  M2 

+  (nic2 


Nil2  Cl  ^ 
£2  ) 


(107) 


where,  based  on  Assumption  5,  /31,/92  E  R1  are  two  positive, 
bounding  constants  chosen  such  that 


Pi  >  II^H  +  I^F- XiFd  ||  ?  (108) 

P2  >  ||JVi2||  (109) 

and  kpUkrU  kp2 ,  kr2  E  R1  are  positive,  scalar  constants  such 
that  kp  —  kpi  +  kp 2  and  kr  =  kr  1  +  kr 2-  After  applying  the 


non-linear  damping  argument  from  [12],  the  bracketed  terms 
in  the  above  expression  can  be  upper-bounded  as  follows 


IHICi  (i 

MCA 
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ei, 

Sl  J 

NIC  (i 

NIO^} 
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£2  J 
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kp2 

IMKIdd 

~kP 2  ||eP||) 

< 
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-  kr2  |M|) 
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(110) 


From  (107)  and  (110),  V(t)  can  be  upper  bounded  in  the 
following  manner 

V  <  -kpi  1 1 ep 1 1 2  -  Ki  \\r\\2  -  kv  \\r]\\2  +  £  (111) 

where  e  =  £i+£2  +  t^  +  t^  G  R1.  The  solution  to  the  above 

Kp2  rSr  2 

differential  equation  allows  us  to  upper-bound  the  tracking 
errors  in  the  following  manner 

||ep(t)||  >  \\ee(t)\\  <  V{t)  <  ai  exp  (— a2t)  +  «3  (112) 

,  T  r  f  (\\  mm(kpi,kri,kn)  , 

where  a±  =  V(0),  a2  =  - - - — V’  anC*  = 

max  (- ,  d* ,  A max  (M)  J 

—  are  positive  scalar  constants,  and  Amax(M)  E  R1  denotes 
<a2 

the  largest  eigenvalue  of  the  inertia  matrix  of  the  UAV. 

We  now  proceed  to  establish  the  boundedness  of  all  sig¬ 
nals  in  the  system.  As  evident  from  (106)  and  (111), 
ep(t),r(t),eo(t),r](t)  E  £00 •  From  Assumption  5  stated  pre¬ 
viously,  and  from  (83),  (88),  (92),  and  (93),  it  follows  that 
XjF(t),ee(t),vfF(t)  E  £00,  and  from  (101),  lVi2(t)  E  Coo-  It 
can  be  observed  from  (1)  that  xTIF(t )  E  Coo-  From  (100), 
IVn(-)  E  £00,  and  hence,  from(54),  Ci(‘)  £  £00  •  Therefore, 
U(t)  E  Coo  (see  (103)),  and  consequently,  Un(t)  and  U(t) 
are  all  bounded  signals.  Hence,  it  is  evident  from  (46)  and 
(99)  that  un(t),ufF(t)  E  £00,  and  ui(t),0c{t)  E  Coo-  We 
have  now  shown  that  all  signals  that  define  the  time  deriv¬ 
atives  of  r(t),ep(t)  and  ee(t)  are  bounded.  Hence,  it  is  easy 
to  see  from  an  examination  of  the  time  derivatives  of  (93) 
and  (103)  that  vfF(t),U(t)  E  Coo-  Hence,  Un{t)  E  Coo-  The 
dynamics  of  rj(t)  is  the  same  as  in  the  leader-follower  case, 
and  it  is  evident  that  JV22(-)  E  Coo-  Hence,  it  follows  from 
(55)  that  C2(*)  £  Coo-  We  therefore  conclude  from  (104)  that 
Ff  (t)  E  £00,  and  from  the  UAV  rotational  dynamics  in  (4) 
that  ufF(t)  E  Coo-  Hence,  all  signals  in  the  system  remain 
bounded  during  closed  loop  operation.  ■ 


Simulation  Results:  The  UAV  mass  and  inertial  pa¬ 
rameters  were  selected  as  given  in  (68)  and  (69),  respec¬ 
tively.  The  simulated  vision  system  is  similar  to  that  de¬ 
scribed  for  the  leader-follower  case,  except  that  the  feature 
points  were  assumed  static  and  located  on  the  ground  plane. 
The  desired  trajectory  was  a  25  meters  radius  circular  path 
in  the  XY  plane,  10  meters  above  ground  (z  =  —10  m). 
The  UAV  initial  position  and  orientation  angles  were  set  to 
[  —  1  1  —  1  ]T  (meters)  and  [  0.01  0.01  1.57  ]T  (ra¬ 

dians),  respectively.  Through  trial  and  error,  the  various 
control  gains  were  selected  as  follows 


kp  — 


UAV  trajectory 


Figure  8:  The  desired  (□)  and  actual  trajectories  (A). 


£1  =  1,£2=  0.1  (114) 

5  =  [  0.5  0  0.5  ]T.  (115) 

The  desired  and  the  actual  trajectories  are  depicted  in  Fig¬ 
ure  8.  The  error  signals  ep(t)  and  eo(t)  are  shown  in  Figure 
9.  From  Figure  10,  it  can  be  seen  that  the  control  inputs  to 
the  UAV  remain  bounded  at  all  times.  Similarly,  the  PTCU 
angular  velocity  input  is  shown  in  Figure  11. 


Appendix  II:  Time  Derivative  of  Auxiliary 
Control  Signal  Un(t) 


From  (50),  the  auxiliary  control  signal  Un(t)  is  given  as  fol¬ 
lows 

Un  =  B^B^U  (116) 

and  the  known  and  unknown  components  in  its  time  deriv¬ 
ative  can  be  written  as  follows 

Un  1  =  B^B^th  +  B^B^U  +  B^B^U,  (117) 
Un2  =  B^B^U*  +  B^B^U  (118) 

where  B^{t)  and  Ui(t)  denote  the  known  terms  in  the  time 
derivatives  of  H1_1(t)  and  U(t),  respectively,  and  similarly, 
B^(f)  and  U2(t)  denote  the  unknown  terms.  We  will  now 
show  the  expressions  for  each  of  the  terms  in  the  equations 
(116),  (117)  and  (118)  for  the  leader-follower  case  presented 
in  the  body  of  the  paper,  and  the  trajectory  tracking  devel¬ 
opment  provided  in  Appendix  I. 

Leader-Follower  Case:  The  inverse  of  B\{t)  and  R2(£) 
defined  in  (42)  and  (43),  respectively,  can  be  expressed  as 
follows 
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—  Rc  03x3 
03X3  —RcCuj1 


20,  kr  =  2,  kp  =  20 


(113) 


(119) 


control  torque  (y)  (N-m)  control  thrust  (N) 


Figure  11:  PTCU  angular  velocity  inputs. 


Figure  9:  UAV  position  (ep(t),  dimensionless)  and  orien¬ 
tation  (eo(t),  m-rad.)  tracking  errors. 
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(121) 


where 


L^1  =  hx3  +  f  sine2  (f  )  5(/x)  +  (l-sinc(0))  S(/n)2.  (122) 


The  time  derivatives  of  1  (t)  and  B2  1  (t)  are  as  follows 
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Figure  10:  Control  effort  for  the  trajectory  tracking  UAV. 
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The  expression  for  1  (£)  and  its  time  derivative  are  given  in 
(120)  and  (125),  respectively.  Similarly  the  time  derivative 
for  the  control  signal  U(t)  defined  in  (103)  can  be  obtained 
by  substituting  the  time  derivatives  for  ep(t)  and  r(t)  from 
(90)  and  (95),  and  expressing  them  in  terms  of  known  and 
unknown  terms  Ui(t)  and  C72  (^) ,  respectively,  as  follows 


C7i  = 


where  Rc(t)  is  given  by  (10),  and  the  time  derivative  of 
Lj1^)  is  expressed  as  the  sum  of  known  terms  L~^(t)  and 
unknown  terms  L~ \  (t)  as  follows 

t>Zi  =  S(nnT eei)  [(sinc(^)-|sinc2  (§))  hx 3 
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2Ci 


-  [kr  +  ^  (Nn  +  N 12 )  -  -^Ci2r  (137) 
V  £1 J  £1 

where  ei;L(£)and  £12(t)  are  fhe  known  and  unknown  terms 
in  the  time  derivative  of 


Similarly  the  time  derivative  for  the  control  signal  U(t)  de¬ 
fined  in  (57)  can  be  obtained  by  substituting  the  time  deriv¬ 
atives  for  Xh(t)  and  r(t)  from  (34)  and  (45),  and  expressing 
them  in  terms  of  known  and  unknown  terms  Ui(t)  and  U2(t), 
respectively,  as  follows 
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U2  —  —  +  'S'  R%CL*  —  v?L^j 
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where  C11(^)and  ( 12(t )  are  the  known  and  unknown  terms 
in  the  time  derivative  of 

Trajectory  Tracking  Case:  The  inverse  of  B\(t)  defined 
in  (97)  can  be  expressed  as  follows 


Bk1  = 


I3  X  3  03x3 

03X3 


-RcLZ1 


(132) 


where  Lu;1(t)  is  defined  in  (122).  The  time  derivative  of 
B^i  (t)  is  as  follows 
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where  Rcif)  is  given  by  (10),  the  time  derivative  of  Lu;1(t) 
is  shown  in  (127),  and  eei  (t)  and  ee2(t)  are  given  as  follows 
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